{ 



ED 183 ~ 951 

AtJTHOR 

IHJSTirOIION* 
PUB MTE . 
NOTE • * , 



Zw i I l/i n g e*r. S£ev en W. ; And Others 
• Z om pe'te-nc v*^Sed Voct ti-onal ffduca tion . 
Office o £ /jSd u cat i on ( DH EWV, W as hin gtpn, D.. C . 
Feb 80 

| 1Api.: Paper presented at the International learning* 
'•Technology Congress and Exposition (3rd* Washington,. 
V DC, February, 19J50) 



BOSS .PRICE 
DESCRIPIDH-S 



• MF01/PC01 Pins Postage, 

Accountability: *C$mpe ten cy Based Education;? * ' 
... r - ♦Educational Research; Educational Trends; Federal 

^Programs; *Hinimum Compe tency Testing ; Research 
* 'Projects:: , *Vocational Education ' 

ABSTRACT . . v . ; * • \ ' , .* ' *' r . . • - > .. ' «. 

* Due to an increased a bareness of* the need for 
accountability in vocational eancation, ' much attention* i^j|eing 
'focused" upon competency- b ased vocational e.ducatioji v *The ySK Of 
of Education, is s^p^ort in g work, for fehje develop »ent^ of co^'|®ten 
measures -for vocational skill areas. Two "occupational coapetenc 
tests in each of seven areas will be developed, reviewed* field 

.-tested., and validated to" ensure their ef fectiveness. These test 
available to all "states that wish to utilize^ them., fiany feature 
compejtency-ba^ed testing, procedures offer advantages over, other 

•. -methods. *These advantages include' bei'ng free 4 from sex- i^ias, and 
' bias .* The identification of . compe te'nc y re.quirement s .f or occupat 
^r'eis is a process which*requires a great deal pf care. 

. CompeteAy- based vocational education will be an increasingly 
impprtorc component in educational programs. The Bureau of 

' occupational and Adult Education supports and encourages its U3 
(Author* : • /- , . 



f. ice o 

-y 
y 

s are 

s. of 

rice * 

i'Snal' 



**********'************************.*** ****************'****************** 

*"v 'Reproductions supplied by |BPS are/the best that can be aaia '* 
* 4 . ,'♦ , ' from the ..original document. ' r ' * 

***'********************************** ************ ***#*,*** ****** ******** 



ERIC 



\ 



UJ 



r 



COMPETENCY BASED VOCATIONAL EDOCATION 



V 



Presented at the Third International Learning 
Technology Congress and Exposition >• 
Washington, D«C» 
• February 1980 



O 



,0j 



\ 



> 



• "Steven W. ZwlIIinger 
Glenn C. Boerrigter 
Howard F. Hjelm . 

United States Office of Education 



2 • 



U S DEPA*TM£NTOF HCALTH. 
? EDUCATION 4 WCLFAfKC > 

" NATIONAL INSTITUTION 

education . ... 

this document mas b€c^ r£pro* 
duced exacilv as r^ccivjed from ^ . 
the person or orGani2At»6n Origin* 

At I HO itT POINTS OF VIEW.6R Q&lNtONS 
SI AT ED DO MOT NECftsARlLy R€PR€* . 
SEN! OPPlCtAL NATIONAL INSTITUTE QP 
EDUCATION POSITIONER 'POLICY 

i » , » 



The competency^ based testing movement has brought this basic concern into 
public view... The passage of the 1976 amendments to the Vocational Education Act 
gave, actional visibility to. the problem of the evaluation of vocational education. 
Among the expanded. evaluation requirements in these amendments are annual state 
accountability reports, .follow-up studies by the 57 States and. outlying areas using 
criteria of employment in training-related occupations, and employer appraisals of 
the extent to which students are well-trained and prepared for employment. 
Additionally) States are required to conduct an evaluation of tndividual vocational 
education programs across the State over *a five year period. 

The introduction of competency based testing in vocational education programs, 
4 Where the subject matter may be more clearly defined than in general education 
•programs, can allow greater guarantees of the effectiveness of the teaching. and of 
the students' learning. . * 

.... ■ ' '" ! ' ■ v ■ ' . • 

Also, competency based vocational education can be used to ensure that 
vocational education programs are evaluated adequately. Competency based testing 
of the completers, of vocational education programs serves to provide evidence that 
the graduates have knowledge of,, and can perform .successfully in, a selected 
occupational area. - ^ 

• Programs which certify their graduates by examination are commqn to many of 
the professions. As more and more occupational aYeas, such as cosmetology, barbers, 
electricians and others are having -State boards of Certification and licensure 
.^established, 4t is-only natural to expect that there would be a corresponding increase 
inthe use of qualifying examinations for program completers in vocational areas* 

It appears that the continued development-of basic competency measures would- 
. answer a serious need in the f ield of ' vocational education. It is not only the 
taxpayers, teachers, and students who want and need, to know how effective, 
vocational education is but others as well. Representatives of . industry, labor, the 
military, and. State departments of education have voiced an interest in tests that 
would accurately reveal the types, nature, and extent of job-related competencies 
acquired by vocational eduction students. • 



. « * ' • ABSTRACT ... » * 

Due to an increased awareness of the need for accountability in vocational 
education much attention is being focused upon competency based vocational 
education. The U»S. Office of Education is supporting work for the development of 
competency measures for vocational skill areas.. , Two occupational competency 
tests in each of seven areas will be developed, " reviewed, field tested, arid 
validated to ensure their effectiveness. These tests are available to all States that 
wish to utilize them. Many features of competency based testing procedures offer 
advantages over other methods. These advantages include Wng free from* sex bias 
and race bias. The identification of competency requirements f or occupational* 
areas is a process which requires a great deal of care. Competency based 
vocational education will be an increasingly important component in educational 
programs. The Bureau of - Occupational and ^Adult. Education supports and 
encourages its use. 

. ' \ . ' . " INT RODUCtlON 



- The United States Office of Education has the responsibility for administering 
educational programs and implementing educational policy as legislated by the 
Congress. Within the Office of Education, the bureau of Occupational and Adult 
Education administers those programs that deal with vocational and adult 
education. The Division of Research and Demonstration administers, the national 
discretionary programs in vocational education. 

• As representatives of the Division of Research and Demonstration, we 
welcome the opportunity to pr.ese,nt„to this group information about our work, our 
ideas, and our goals in regard to competency based vocational education. 

•< - N • • \ * - 

More specifically, we will provide information about work which is being 
supported in the area of competency based vocatibnal education and applications 
for competency based vocational education. 

. CALL FOR ACCOUNTABILITY ' ( . ^ 

There has been an increased measure of importance attached to competency 
based education. This is due, in part, to the current demand for accountability in 
education. Public acceptance of expenditures for education is becoming more 
contingent upon the school's "track record". '• ,In brief, good education will be 
supported, ineffectual education programs will not, be supported. Recently many 
questions such as the following have been asked about vocational educations Is the 
expenditure of public funds justified? Are there employment opportunities for the 
graduates of vocationartraining programs? Is vocational education effective? (1) 

- _ Op to this time, due to the lack of other 'measurable criteria, the major 
criterion concerning the effectiveness of vocational education has been the 
placement rate for graduates. While the placement rate of graduates is reflective 
of a . great many things; regional and local employment conditions, general 
economic activity, or the willingness or unwillingness of local employers . to accept 
a diploma as proof of a job skill, it does not in any consistent manner attest to the 
effectiveness of the teaching that the students have experienced or the* skiHs they- 
have learned. Teachers, as well "as employers, need something to, measure their 
effectiveness. , How can a teacher determine if a student has made adequate 
progress? Without some objective guide, a teacher has no choice but to 
subjectively compare the student's behaviorial patterns with pre-stated objectives 
and must rely upon his/her own judgement. The essential question is: To what 
degree are these judgements valid? * . . 

- V , V ... 4 
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, ." U.S. OFFICE OR EDUCATION SUPPORTED PROJECTS 

American Inst itutes for Research ' • 

The Office of Education ., has recently. awarded a three year contract for the 
t development of competency measures for vocational skill areas. The contractor will 
develop and validate tests that will allow the measurement of competencies acquired 
by students in selected vocational education programs. \ 

- ". ' . " " ^ *' 

. • ..This project also reflects the importance that the National Association of State 
Directors.^of Vocational Education attaches to this area. They requested, in "response 
to, a call for comments published in the federal * Register , that the Office of 
Education issue a Request for Proposal: for the support of a project to develop 
competency tests. Each year the Bureay of Occupational, and Adult Education 
publishes in the Federal Register its proposed activities and priorities" for the coming 
'year. . The Bureau asks for suggestions and comments from the interested public in 
response to this announcement. x ■ • 

The American Institutes for Research, the contractor selected by the Office of 
Education, will develop selected occupational student competency tests for use in 

.. vocational education programs. Two tests will be developed in each of the following 
seven program areas': (i) trade and industry, (2) home economics, (3) health, (4) 
distributive education, (5) technical, (6) business and office, .and (7) agriculture. 
These, tests .wild include evaluation methodologies for' measuring students' 
performance on selected - tasks. Also, methodologies for measuring the students' 
cognitive skills will be, included in order to measure their knowledge and 
understanding of the subject. The students' affective outlook, their attitudes and 
motivation towards the occupational area, will also be able to.be evaluated. These 

\ tests will be thoroughly reviewed, pilot tested on a small scale, 'revised if necessary, 
nationally field, tested, and validated to ensure their, effectiveness. Field testing and 
validation will include data obtained both from students in school and employees in 
industry. - « ' ' V 

Foliow&g completion of the developmental work, a. national workshop will be 
held to give a full briefing on the tests to State education agency and postsecondary 
personnel in all states and outlying areas. Project staff will also provide technical 
assistance, to selected States if the States 'choose to implement the tests in thejr 
regular school programs. Project staff will also develop and help to implement a plan 
for the continued .development of occupational competency testing on a self- 
supporting basis after the three year contract is completed. ; x 
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It is important to bear* in mind that although these tests will be develQped*with 
support from the Federal" Government, these tests win no^in any way, be Federal - 
tests.. This will not be a Federal testing program. The tests will be made available to 
States and educational institutions which can then us^them if they wish. No one is 
required to use these, tests. However, we belie venhat these tests will be., of high 
.enough quality, that States, local", education/^igencies, individual- schools and 
educational organizations will want to use them. . 

Competency based testing Is. noj/hew to the field fcf vocational education. 
Vocational/educators have used. competency based testing successfully for many 
years. fXn outstanding example oi^this is in the area of business education where 
typing and shorthand tests andjtests involving office equipment bave been used ^or a 
considerable period of time. • * , : 

The National Center forResearch in Vocational Education 

Other efforts to jtevelop* performance based competency materials have 
received Federal- support. The National Center for Research in Vocational Education 
at the Ohio State University has developed Performance Based Teacher Education x 
Materials. Performance based teacher education is an approach to teacher 
preparation in which' the teacher is required to demonstrate essential teaching tasks 
in an actual teaching situation. Actual performance of the tasks helps to ensure that 
the teacher has not only the "knowledge required, but. also the ability to perform the 
skills that; are essential to successful teaching. Perfdrmance based teacher education 
is a promising alternative to the more traditional preservice and inservice methods of 
educating teachers* 

One hundred modules in ten teacher education professional skill categories have 
been produced and the response from the 'vocational educational field has been 
overwhelming. AH . 50 .States, all Canadian provinces, 15 foreign countries, 5 
departments of r the United States Government, over U00 colleges, and 200 local 
education agencies, as well as private corporations such as Kodak, 3-M and IBM have 
purchased these instructional materials. The* office of \Educati on is currently 
supporting efforts by the National Center to develop additional modules to adequately 
prepare teachers for working with special needs populations. 



National Occupational Competency Testing Institute / * • \ . * 

. Another organisation which" is very involved with vocational, education 
competency testing is the National Occupational - Competency Testing Institute 
(NOCTI). This organizatMlH|!evelops performance based examinations which are 
based upon task analyses of selected jobs. These task analyses are used for the 
development of tests for the assessment of • skill . competencies of teachers' for 
certification ^purposes by the. States -as well as for the. development of student 
competency examinations, . The group~that became The National Occupational 
Competency Testing Institute >as originally funded by a grant from the Office of 
Education and has since evolved into'a very effective organization working with 47 
States. This Institute engineers highly reliable instruments which can be used to 
assess the quality of teachers, students, and/or programs. * . *, 

* r 

\ ' ; FEATURES OF COMPETENCY BASED VOCATION Al,. EDUCATION f \ 
• .*••„.•. V , . _ * . * , 

* . There -are major advantages to t be gained from thex use of competency based 
testing procedures. These advantages include allowing the direct measurement of job • 
skills which cannot 'be interred from written tests alone, and they can provide a high 
level of realism both in the tasks performed and in the surrounding conditions., 

• -Of importance to the long standing but newly jimphasized requirements of 
equity, the verbal demands on the examinee made by sample job tests can more often 
match the. actual job requirements than can paper and pencil tests. * This pr ocedure 
significantly, reduces the test bias against those students with low verbal skills. 
Furthermore, competency, based testing is, by its very nature, largely free from both 
sex bias and race bias. In such testing procedures, competencies override biases. 

► . *• ... 

( Ideally, competency based vocational education programs should encompass the 
following features: (2) • . • 

y • , ' . • 

_ 1. Pre- testing students upon entry to determine the skills they already have asv 
well as determining the competencies that need to be achieved^ 

* » ■ *. * * • 

2. Allowing each student to proceed to subsequent instruction as spon as 
competencies are attained. - 

* 

3. Providing an alternative 'method of instruction if a student does not achieve 
• a competency. 

4. Recording student performance as each competency is achieved. 

5. Placing greater -emphasis on exit requirements (competencies to 5e 
achieved) than on entrance requirements. * ' • 



6. Assessing students on the basis of competencies. 
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Obviously it is this last step, assessing students on the basis of competencies 
that we are most concerned with in our discussion. The effectiveness of competency 
based vocational education is largely dependent upen tfce development .and 
appropriate use of valid, competency measures. These measures are needed for 
^certifying student proficiency and for diagnosing students' needsj but they are also of 
importance far satisfying any evaluation requirements, that Have to be met. 



'DEVELOPMENT OF COMPETENCY/TESTS 

. • ••• v- v •; • . . if • 

The difficulties of constructing, administering, add scoring competency tests 
are well kno>vn. Usually \these tests are based upon eitfcer job analysis to determine 
which skills and abilities are of chief importance in the lob or they are based upon a 
family of behavioral objectives. The process of selecting the tasks thestudents must 
K2 Qnn md the, condition^ under, which they shdbid be performed is the single most 
difficult part of competency based vocational education. Evaluation criteria must be 
developed so that a teacher ^s able to systematically score a student's performance. 

t Although a prospective, bricklayer can be asked to lay a wall of bricks in a 
straight line and thus, provide an indicator of how successful he(she) may meet one 
job. entry skill required for the, field of bricklaying, there are other occupational areas 
in which the selection of tes^ criteria is not so easy. How, for instance, do you 
.develop suitable performance exercises for determining job entry skills for an area 
such as firefighting? r 

The. identification of competency, requirements for occupational areas is a 
.process which requires a great deal, of care .and raises many questions. Among the 
major concerns are the following; {3) 

: \ . ' ' 

Frequency. - Wh%t proportion of the workers in the occupation actually 
perform the task? Mow often is the task performed? 

2. Importance - Does the task represent a critical 'aspect of the job? Are other 
tasks dependent upon it? .,.'.*'* 

3? > Difficulty - Is the task performed by entry level workers? Or does it require 
advanced practical knowledge?* / .. s - 

\ Complexity - Does the task subsume a number of simpler task elements? * » 

5. Expectations - Dp employers assume the competency *to be learned in a 
, school or is it learned on the job? 

^6. Job Success -Is there evidence (or consensus among emplpyers) that persons' 
having, particular competencies are more |ikely to be successful qn the job? 

• 7 » Testing Domain ^- Does the task fall primarily in the- affective, cognitive, or " 
t psychomotor domain? % 

" 8 * Administration of Testing V Can performance of the task.be adequately 
assessed m schools, given the requirements of time, equipment *nd safety? 

^ Generality • Is. the task performable by anyone or are certain categories of 
people at a disadvantage? [ ./ 

10. 'Availability of Tests - Are there validated, nationally used tests? * r 

!*• Certification Requirements - Are there business, governmental, or union 
certification or requirements involved? 



This list of concerns makes\it clear that a thoughtful and careful analysis of the 
tasks In each occupation is essential if competency based tests are to be a useful 
'method of measuring a student's performance. The fourteen occupational 
competency tests to be developed by the American Institutes for Research will be 
constructed in accordance with the 1 con'cerns listed above. • • » 

. * ; . • ; . \ • .. • • : • 
.Suppose, that a woodworking vocation has been selected and the critical tasks. 
Which should be tested involve, as a task element, the use. of a circular saw to cut 
boards and the. accurate measurement of the width of* the cut; The question could 
arise as to whether this task/item would inherently prevent a visually handicapped 
person from demonstrating competence. The ^answer lis that it may not exclude a • 
visually handicapped person who had access to" appropriate adaptive equipment. In 
this example, the measuring 'aspect could be handled) either through the use of a 
specially marked metal rule, a special "click-rule" Available from the American 
Foundation for the Blind), or through a shop-made gauge block. («> . " ' • • 

• • During the task analysis, if it. appears that a certain task, unavoidably, presents 
.systematic difficulties for certain populations, this will ' be noted and possible 
exception rules or adaptations will be suggested, by the American Institutes for 

Research. V • 

I . . . 

A question that is often raised is should the scoring of performance tests be 
based on process or on product ? In general, when^ a physical product is the goaf of the 
work and if standards of quality ace available, i.e., -accuracy, meeting specified 
tolerances, etc. the test should be designed to provide information about the product. 
A simple illustration of product evaluation drawn from woodworking would be the 
eValuatiqn of a board which has been cuf and finished in conformance with' a set of 
directions. A more complex example would be the evaluation of a cabinet or bureau 
made from plans. The evaluation, in tftis,case, would be based upon its conforming to 
prescribed dimensions, the tightness of the joints, its sturdiness'and its finish. 

In those cases where the end product is not a standard item, or where the result * 
is a cooperative effort of several workers working together, or where adherence to 
safety precautions are critical, then competency tests should measure the adequacy 
of the process. (5) * ' 

'•' • « » . » . , 

One of the strengths of process evaluation is that it can * be diagnostic of 
specific difficulties the student may have. Deficiencies such as misinformation, 
wrong choice of tools, and inadequate skill level are obvious if competency testing is 
used in an intermediate check on progress and use can be made of this information in 
. .Imping, ^e studenV..Oi course, in. some cases a comianatiorj of product. and process 
evaluation is necessary. ' - 
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THE.FUTURE OF COMPETENCY BASED VOCATIONAL" EDUCATION ' 

Competency based education is a valuable tool. It is one of *the many tools 
.available to educators in general and to vocational educators in particular ^ Although 
there are many applications and numerous benefits ho be gained from wider 
recognition of the efficacy of competency based vocational education programs, we 
must keep in mind that it is one of many tools. $ is not a solution for everything. 
Even , though we are putting considerable sums of money into/competency test 
development we would be among the first to admit/that it is not a solution for all 
: difficulties encountered in vocational educations * - 

As pur society advances we will hear increased demands for. "proof" that our 
schools work. We will see the wide-spread use of modularized curriculum, in which 
learners progress at their own rate after demonstrating the successful completion of 
earlier modules. Students will demonstrate their mastery of the modules as they 
\progress by showing; themselves competent. Their performance on tests will indicate 
they have acquired the skills needed to advance, * 

As education moves toward open entry, open exit systems, we will no longer 
automatically promote learners every 3une. at the. end of tne school year. Students 
will be able to start at any time and when they demonstrate competence in the skills 
of the subject matter, they will not have to* continue in the class.. » 

* * » * 

An awareness of the directions in which 4 our schools and educational techniques 
are moving. demand that we prepare for these changes. That is what the Office, of 
Education is trying to do with its support for the development of competency based 
performance "tests. • *"''•'"■ 

There are many positive aspects to competency based vocational education. In 
addition to. the advantages mentioned earlier such as being freedom both sex ana" 
race-^as and providing a high level of realism for testing purposes, there are other 
benefits to be gained. These advantages include the fact that competency based 
vocational education has demonstrated over and over again' that the students exposed 
to it^meet their learning objectives. , % * * , ^ 

',. * « ' ■ • * 

Competency based testing allows observation of the skills that have been 
: mastered by students as well as laeing useful for the diagnostic evaluation of students' 
^performance. These tests are specif ic enough so that the results are more objective 
and reliable than other evaluation methods such as ^written achievement *tests and 
, subjective performance ratings.. > * * • N 



* i *• . . . , • • • i 

Obviously all behaviors cannot be tested. Work samples and simulation tests do 
not equal the actual workplace. . The assumption underlying tnt use of the work 
sample .is tfiat its dose approximation of the circumstances and pr6cedures of the 
real workplace. present a truer, measure of the students 1 competence at the appointed 
task. Work, samples are also used to measure occupational competence in ways that 
paper and pencil tests cannot. For example, the*presentation of. a malfunctioning TV 
set to. a student in a TV technician course measures those*com potencies fy a t paper . 
and penal tests cannot. As mentioned earlier! student typists and sllenography- 
students work almost exclusively with work sample/. . ' ' . 

.' . " * , t . 4 ; " ... 

There are some problems with competency based testing that have* to do with 
the actual test itself. The tests may require the use of. scarce equipment, producing a 
logistics problem. They mav^ require the use of equipment that is-delicate and easily 
damaged and is expensive 10 repair or replace. Additionally; the use of some 
equipment may present hazards for the student, .particularly if the student's 
competence cannot be demonstrated satisfactorily. 

For every occupational area for which competency based vocational education 
is. desirable, -task analyses have to be conducted. Various behaviors have to be 
identified. Operational criteria -must be . selected. The possibility of the • 
technological obsolescence of ^particular jbb fields must be considered, 

; Education has to be provided for vocational educators so that they may learn 
>«Ow competency based vocational education was developed and how they can use it if 
they desire. A Jarge information dissemination program, of which this presentation is 
one small piece, is needed to increase awareness of these techniques. 

....... 

We must not underestimate the difficulties of identifying appropriate scoring 
standards. A* great deal of time and energy must be devoted to this area in order to 
arrive at valid tests and scoring systems*. In the American institutes for Research 
project, scoring standards for four discrete areas will be developed. Content related 
to such areas' of job performance as (a) operations (skills involving the correct 
manipulation of tools and equipment to complete tasks; using correct procedures), (b) 
related technical* information (necessary to carry out the tasks), (c) trade theory 
(technical background -knowledge associated with , the tasks), and. (d) general 
knowledge (safety considerations, etcO will be included in these standards*. 
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CONCLUSION 



» Our; position Within the Bureau of, Occupational and Adult Education is, 
obviously, that we believe that competency based vocational education is a very 
valuable tool whose use should be more widespread than is at present.' We encourage 
its development and our 'support of the American Institutes r «for Research project is 
the harbmger , of expanded interest in this area. We anticipate continuing 
encouragement and support of competency>ased vocational education. We hope that 
the State Directors of Vocational Education and other influential decision-makers 
within-- the States will continue their' support ^and interest in competency based 

vocational educations , ' "* * i 

' » ■ % . • . . ».•' 

• 1 ' ' V . ' • '. * 

Competency based vocational education is an effective tool and we support its 
spread. ' We are' in favor of a systematic, approach to testing which, "uncovers" 
-students' proficiencies' and which improves accountability of Vocational education 
programs. In summary, Jet us indicate v t hat with" the advent of competency based 
vocational education a need has been created for occupational competency measures 
well beyond those that are now available. The effects and benefits ^Vocational" 
student competency testing are far-reaching. The/Yesuits will be valuable in 
revealmgjo both teachers and students the progress that has been ma* in preparing 
Students for specific jobs, as well as in certifying student competencies to" 
prospective employers. Moreover, competency testing w^ll have an increasing impact 
in the ^Jture on the area of . program improvement. Such testing should yield results 
« useful to vocational administrators, teachers, guidance and counseling personnel and 
others concerned with the further improvement of the teaching and learning process 
in vocational education. ^ 
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